Upgrading Airflow: 2.x to 3.0
Smaller Than It Sounds
Airflow 3.0 is a major version bump with real architectural changes (the Task SDK, DAG Versioning, Assets). For most existing 2.x DAGs written using the TaskFlow API and modern idioms already taught throughout this course, the actual migration is closer to a rename-and-test pass than a rewrite.
What Almost Never Needs to Change
If a 2.x codebase already follows the patterns this course teaches as default — TaskFlow API, airflow.providers.standard.operators.* imports, Datasets — most of it carries forward with minimal changes:
| Already using... | 3.0 status |
|---|---|
@dag / @task (TaskFlow API) |
Unchanged in spirit; import path moves to airflow.sdk |
Dataset |
Still works via a compatibility alias to Asset |
PythonOperator, BashOperator |
Unchanged behavior |
TaskGroup |
Unchanged |
| Provider packages (Amazon, Google, etc.) | Each has its own 3.0-compatible release; check the specific provider's changelog |
What Actually Requires Changes
Import path updates. The clearest mechanical change:
# Airflow 2.x
from airflow.decorators import dag, task
from airflow.datasets import Dataset
# Airflow 3.x
from airflow.sdk import dag, task, Asset
DAG-level code execution. Airflow 3's Task SDK enforces a stricter separation between DAG-parsing-time code and task-execution-time code — any DAG file already following this course's repeated advice ("no heavy logic at the top level of a DAG file, only inside tasks") is already compliant. DAGs that violate this are exactly the ones the Common Pitfalls module already flagged as bad practice for unrelated reasons — fixing pitfalls first tends to fix migration blockers too.
Removed deprecations. Airflow 1.x-era patterns still technically working under 2.x with deprecation warnings (SubDAGs, some legacy operator import paths) are fully removed in 3.0. Running airflow dags list and checking for deprecation warnings under 2.x before upgrading surfaces these ahead of time.
A Practical Upgrade Sequence
- On the current 2.x version, resolve every deprecation warning first — this is safe, reversible work that doesn't depend on 3.0 at all.
- Upgrade provider packages to versions with confirmed 3.0 compatibility, checking each provider's own release notes.
- Stand up a parallel 3.0 environment (not an in-place upgrade) and point it at the same DAGs folder — Airflow 3.0 can parse most 2.x-era DAG code as-is, letting you see real parse/import errors before committing.
- Fix what the parallel environment surfaces, then cut over.
Validate against a real, running parallel environment before promoting - never upgrade the thing serving real traffic first and find out what breaks.